Research Brief 
Center for the Advancement of 
Engineering Education 
NSE Grant ESI-0227558 



CENTER FOR THE ADVANCEMENT OF ENGINEERING EDUCATION 


What’s So Important About Peer Review of 
Teaching Portfolio Components? An Exploratory 
Analysis of Peer-Review Episodes Within ETPP 

Authors: Jennifer Turns, Jessica Yellin, Yi-Min Huang, and Kathleen Gygi 
Source: 2006 American Society for Engineering Education Conference 


Understanding and promoting effective teaching are central concerns of the engineering 
education community. The following summary reports on research to investigate the processes 
by which construction of teaching portfolios in a socially supportive context can promote the 
advancement of teaching knowledge and ability. By characterizing how a specific intervention 
can advance teaching knowledge and effectiveness, findings and ideas are generated that can 
help others engaged in the same goal. 


Any approach to advance teaching will likely need to 
address two issues. Eirst, the approach will need to be a 
learning event where teachers succeed in learning 
something (e.g., new techniques, new ways to think about 
students) that will help them improve their teaching. 

Second, the approach will need to fit into the complex 
schedules and contexts of educators (for example, real- 
time problem solving or preparation of tenure review 
materials) since simply spending time learning about 
teaching may not be possible. This suggests two questions regarding helping educators become 
more effective: a) to what extent does the approach lead to learning knowledge and skills related 
to teaching, and b) what outcomes other than learning about teaching do participants derive from 
the approach? 


Peer review of teaching portfolio 
components provided a context for 
helping participants improve their 
written products, grapple with the 
affective nature of talking about 
and sharing teaching ideas, and 
learn knowledge and skills relevant 
to teaching. 


Implications of Findings 

These results suggest support for claims that the peer review was much more than simply a fuzzy 
experience, but rather provided a context for helping participants improve their written products, 
helping participants grapple with the affective nature of talking about and sharing teaching ideas, 
and also helping participants learn knowledge and skills relevant to teaching. 


Method and Background 

This study looked at the CAEE Engineering Teaching Portfolio Program (ETPP) as a learning 
event — how the individual and social processes associated with building a teaching portfolio 
could lead to advancement of teaching knowledge and ability. Additionally, this work focused on 
the strategy of having future engineering educators (graduate students) create teaching portfolios 
through a scaffolded process and in a group-oriented environment. Participants in the ETPP 


prepared a teaching portfolio consisting of a teaching philosophy, two to five annotated artifacts, 
and a diversity statement. Peer review — defined as reciprocal evaluation of written products — 
was originally conceived as a core element of the program and makes up about half of each 
ETPP session. 

Evidence suggests that the peer review periods may be important from the perspective of both 
learning and other participant outcomes. This and other indicators identified the peer review 
activities as one of the most important elements of the ETPP. Developing a greater 
understanding of what is going on during these periods provides insights to programs that create 
approaches to improve teaching. 

The following question motivated the work presented in this paper: what is going on during the 
ETPP peer review sessions that would explain why researchers and participants alike point to 
them as an important element of the program? The analysis covers a subset of the activities in 
four different peer review sessions from the 2003 ETPP offering selected to cover a range of 
issues. This offering consisted of two groups of eight participants. A comprehensive set of field 
notes were collected that aimed to capture as much of the session interaction as possible. The 
notes were coded at the level of activity and the team’s understanding of the activities was used 
to make inferences about the two phenomena of interest: learning and additional participant 
outcomes. 

What We Found 

The participants discussed issues in the following topic/coding areas: audience, genre 
negotiation, writing, clarification, sharing ideas, positive reinforcement, stage fright, and task 
negotiation. Based on the analysis, the original two questions were again revisited: 

To what extent does the approach lead to learning of knowledge and skills related to teaching? 
The activities also provided a strong basis for making inferences about the peer review sessions 
as a site for significant learning of knowledge and skills related to teaching. There is evidence 
from these sessions for two types of learning mechanisms: information exchange (new 
information) and knowledge revision. The coding captured instances of participants sharing 
teaching information in both declarative and narrative (story) forms (information exchange), and 
that participants were thinking about how to clarify their ideas, improving their underlying 
knowledge base (knowledge revision). 

What outcomes do participants derive from the approach other than learning about teaching ? 
The coding of participant activity provided a basis for making inferences about the types of 
outcomes other than learning that participants derived from the peer review sessions. Eor one, 
participants were getting information that would lead them to create improved written products, 
something of interest to participants because of the role of these products in the job search 
process. The results also suggest participants had the opportunity to experience a productive 
collaborative writing environment, one that likely differed from prior professional writing 
experiences. The results further suggest that some participants may have had the opportunity to 
increase their confidence in their ideas about teaching and their ability to articulate them. 
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